מבוא כללי לתכנות ולמדעי המחשב

Transcription

1 מבוא כללי לתכנות ולמדעי המחשב מרצה: אמיר רובינשטיין מתרגל: דין שמואל אוניברסיטת תל אביב סמסטר חורף שיעור 6 ייצוג תמונה דיגיטלית מבוא 1. ייצוג תמונות בזיכרון המחשב 2. תמונות סינתטיות 3. עיבוד תמונה בסיסי 4. העשרה 5. 1

2 תמונות דיגיטליות מטרת שיעור זה היא לספק הבנה בסיסית של עולם התמונות הדיגיטליות, מהמסתורין האופף אותו. ולהסיר חלק בסוף שיעור זה: תבינו כיצד תמונה מיוצגת תדעו לייצר תמונות מלאכותיות תוכלו לבצע מניפולציות בזיכרון המחשב עם תבניות שונות פשוטות על תמונות אמיתיות (ומשונות) יחד עם זאת לא ניכנס לסוגיות טכנולוגיות ספציפיות, שקיימות בשוק. או לאופן השימוש בתוכנות 2

3 Brief "Historical" Technological Context - early 1980's - today transistors speed 29 K 4.77 MHz 1.4 G 3.7 GHz processors memory RAM 640 KB 4 GB Hard Disk 5 MB 500 GB communication , simple text (128 ascii chars) tons of data, inc. images (next slide) 3

4 A Brief Historical Context, 30 Years Later With the proliferation of (1) larger and faster memory, (2) strong, inexpensive processors, (3) faster internet, it became possible to efficiently (1) store, (2) process, and (3) transmit large digital images. Facebook stores about 350 million photos DAILY (reported Sep 2013). 1.1 billion photos where uploaded on 2013 New Years Eve. The total number of photos shared on Instagram is 16 billion. On average, 55 million photos are posted daily (reported Dec 2013). (Instagram was launched on Oct 2010!!). On Flickr the average upload of images per MONTH in 2012 was about 43 million. This dramatic technological progress is reflected by the following saying, often attributed (apparently incorrectly) to Bill Gates, in 1981: "640KB ought to be enough for anybody". 4 Slide (modified) courtesy of Prof. Benny Chor

5 Digital Image Representation A digital image is encoded (represented) as a matrix of numbers Each element in the matrix is called a pixel (picture element), and represents the light intensity / color at that location of the image x m columns pixel (0,0) pixel (m-1,0) y.. pixel (x,y) n x m matrix. n rows pixel (0,n-1) 5

6 Gray Level Images For simplicity, we will deal with gray-level images (although what we will see can also be applicable, with minor changes, to other formats). 8 bits per pixel (2 8 =256 gray levels): 0 = black, 255 = white 38, 26, 21, 36, 19, 28, 33, 44, 31, 112, 77, 83, 34, 168, 159, 48, 50, 14, 55, 211, 112, 137, 34, 101, 129, 62, 54, 40, 21, 86, 41, 46, 35, 19, 35, 52, 18, 57, 39, 123, 38, 16, 38, 67, 45, 21, 29, 59, 10, 130, 45, 43, 46, 51, 44, 39, 53, 31, 24, 64, 47, 30, 54, 45, 40, 46, 23, 26, 58, 40, 71, 57, 66, 63, 70, 84, 65, 62, 91, 49, 72, 55, 43, 57, 90, 111, 92, 73, 74, 56, 47, 45, 36, 78, 114, 113, 81, 54, 57, 44 6

7 Number of bits per pixel. Determines number of possible hues (גוונים) per pixel Image Bit Depth Image from: A human observer is able to discriminate between at most a few hundreds shades of gray in optimal conditions (some estimations are lower, depending also on the background, distance from the image etc.). Therefore, 8 bits are normally enough for grayscale images (8x3=24 for RGB). Higher bit depths images are sometimes aimed for an automated analysis by a computer (e.g. CT images).

8 Colors B&W / gray-level / RGB B&W (1 bpp) 256 gray level image (8 bpp) "true color" image (8+8+8 = 24 bpp) Images from: 8

9 - A 256 gray-level image of size 512x512 pixels is stored in a file. What is the volume of the file? Exercise 1) 512 Bytes 2) 256 KB 3) 4 MB 4) 12 GB

10 Images in Python We will use the Python Imaging Library PIL / Pillow. It enables: Reading / storing images Displaying image Basic image processing Installing Pillow: first check which version of Python 3.6 you have: 32 or 64 bit (see figure) Windows users: if you have a 32 bit version, click Pillow win32-py3.6.exe if you have a 64 bit version, click Pillow win-amd64-py3.6.exe (both links are from this website: Mac users: type pip3 install PILLOW in a terminal shell 32/64 bit 10 Upon successful installation, this command should yield no error messages.

11 Basic Handling of Images using PIL from PIL import Image # Open image >>> im = Image.open("./my_image.jpg") "./" = current folder "../" = parent folder >>> im.size (388, 541) #width, height >>> im.show() # display # Save as a new file >>> im.save("./new_image", "bmp") # crop >>> region = im.crop((200,410,210,440)) >>> region.show() # convert to 256 gray levels # so the code we write later will work >>> im = im.convert('l')

12 Example: Rotating an Image in PIL coins = Image.open("./coins.jpg") coins.show() coins_rot = coins.rotate(45) coins_rot.show() 12

13 The Matrix Representing an Image # Load image data into a matrix # Changes in the matrix WILL affect image >>> im = Image.open("./my_image.jpg") >>> mat = im.load() >>> mat[0,0] 31 mat[x,y] is the pixel at column x and row y >>> mat[0,0] = 255 >>> mat[0,0] 255 >>> for x in range(20): for y in range(20): mat[x,y] = 255 >>> im.show() #im was affected by the change in mat! 13

14 Synthetic Images # Creating a new image im = Image.new(mode='L', size=(100,50), color=255) # 'L' for 256 gray levels # size: (width, height) # initialized 255 (all white) def vertical_lines(w,h): im = Image.new(mode='L', size=(w,h), color=255) mat = im.load() for x in range(w): if x%10 == 0: for y in range(h): mat[x,y] = 0 return im 14 vertical_lines(100, 50).show()

15 Synthetic Images (2) def diagonal(n): im = Image.new(mode='L', size=(n,n), color=255) mat = im.load() for x in range(n): for y in range(n): if x-y == 0: mat[x,y] = 0 return im >>> image = diagonal(500) >>> image.show() #??? 15

16 Synthetic Images (3) Additional, rather unexpected examples: def product(n): surprise = Image.new(mode='L', size=(n,n), color=255) mat = surprise.load() for x in range(n): for y in range(n): mat[x,y] = (x*y) % 256 return surprise def circles(n): surprise = Image.new(mode='L', size=(n,n), color=255) mat = surprise.load() for x in range(n): for y in range(n): mat[x,y] = (x**2 + y**2) % 256 return surprise 16

17 Synthetic Images (3) >>> circles(256).show() >>> product(256).show() 17

18 Manipulating Images - add def add(im, k): w,h = im.size new = im.copy() mat = im.load() mat_new = new.load() for x in range(w): for y in range(h): mat_new[x,y] = (mat[x,y]+k) % 256 return new The parameter im is a PIL image object of 256 gray levels. For example: >>> im = Image.open("./my_image").convert('L') >>> new = add(im, 50) >>> new.show() 18

19 Manipulating Images - add >>> im = Image.open("./eifel.jpg").convert('L') >>> new = add(im, 50) >>> new.show() 19 im add(im, 50).show()

20 Manipulating Images (2) - negative def negate(im): w,h = im.size new = im.copy() mat = im.load() mat_new = new.load() for x in range(w): for y in range(h): mat_new[x,y] = return new 20 im negate(im).show()

21 Manipulating Images (2) - negative def negate(im): w,h = im.size new = im.copy() mat = im.load() mat_new = new.load() for x in range(w): for y in range(h): mat_new[x,y] = 255 mat[x,y] return new 21 im negate(im).show()

22 Manipulating Images (3) upside_down def upside_down(im): w,h = im.size new = im.copy() mat = im.load() mat_new = new.load() for x in range(w): for y in range(h): mat_new[x,y] = return new 22 im upside_down(im).show()

23 Manipulating Images (3) upside_down def upside_down(im): w,h = im.size new = im.copy() mat = im.load() mat_new = new.load() for x in range(w): for y in range(h): mat_new[x,y] = mat[x,h-y-1] return new Left-right switch? 23 im upside_down(im).show()

24 Additional Topics as time allows 24

25 Resolution and Pixel Physical Size Resolution is the capability of the sensor to observe or measure the smallest object clearly with distinct boundaries. Resolution depends upon the physical size of a pixel. Higher resolution = lower pixel size. Source: Wikipedia Increasing resolution 25

26 Higher Dimension Images A 2D image is encoded as a matrix 3D: spatial slices of 2D images video 2D images over time 26

27 Compression and Image Formats Digital images with high pixel resolution and bit depth take up lots of computer memory. This motivates the need for compressing images. During compression, some of the information in the image may be lost, in which case the compression is termed lossy. Otherwise, we call it lossless. jpg, tiff, png, bmp, gif etc., differ by the type of compression applied to the original image. The bmp format is lossless, while the other formats are lossy (tiff can be both, depending on some parameter settings).

28 The Example of jpg jpg format partitions the image into squares of 8-by-8 pixels. Most such squares will exhibit only gradual, moderate changes, especially in smooth areas of the image. These gradual changes can be well approximated by far fewer bits than the = 512 bits in the original representation. A factor of 10 (or even more) saving in space can be achieved. original image highly compressed version Human HT29 colon-cancer cells. In the compressed image on the right, all the pixels in the blue square are identical. In the green square, pixels only change from top to bottom. In the yellow square, pixels change in both directions.

29 The Example of jpg Human HT29 colon-cancer cells. In the compressed image on the right, all the pixels in the blue square are identical. In the green square, pixels only change from top to bottom. In the yellow square, pixels change in both directions.

30 Typical operations: 1. Color corrections / calibration 2. Image segmentation 3. Image registration / alignment 4. Denoising (noise reduction) 5. Edge detection Image Processing Typical applications: Machine vision Medical image processing Face detection Augmented reality 30

31 From Wikipedia: Segmentation In computer vision, image segmentation is the process of partitioning a digital image into multiple segments (sets of pixels, also known as superpixels). The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images. For example, applications in medical imaging: - Locate tumors and other pathologies - Measure tissue volumes - Diagnosis, study of anatomical structure 31 Image from:

32 Segmentation by Thresholding Simplest segmentation method Apply a threshold to turn a gray-scale image into a binary image (BW) this is called Binary Segmentation Human HT29 colon-cancer cells Binary segmentation, threshold = 20 Can apply more than one threshold, creating >2 segments 32 The key is to select the appropriate threshold values

33 Binary Segmentation Which threshold is the best? Original Threshold = 20 Threshold = 40 Threshold = 60

34 Binary Segmentation - Code def segment(im, thrd): ''' Binary segmentation of image im by threshold thrd ''' width, height = im.size out = Image.new('L',(width, height)) mat = im.load() out_mat = out.load() for x in range(width): for y in range(height): if mat[x,y] >= thrd: out_mat[x,y] = 255 else: out_mat[x,y] = 0 return out

35 Edge Detection Edge - sharp change in intensity between close pixels Usually captures much of the meaningful information in the image A very useful image processing application images extracted using a Sobel filter from: 35